[core] Make hook.metadata a lazy Promise getter - #3988
Conversation
🦋 Changeset detectedLatest commit: 2d28ade The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
There was a problem hiding this comment.
🟡 Changes recommended
The exported HookWithLazyMetadata.metadata type currently doesn’t reflect that it can resolve to undefined, which contradicts the implementation and docs and can mislead API consumers.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates the core runtime hook API so hook.metadata is hydrated lazily via a memoized Promise getter, avoiding the run fetch + run-key round trip unless a caller actually reads metadata. This keeps hot paths like hook resumption at a single hooks.getByToken read while preserving existing webhook behavior that depends on metadata (respondWith).
Changes:
- Introduced
HookWithLazyMetadataand updatedgetHookByToken()/resumeHook()to return hooks withmetadataas a lazy, non-enumerable Promise getter. - Updated
resumeWebhook()to explicitlyawait hook.metadata(the one resume path that needs metadata) and to reuse the derived read-side key for the payload write. - Added/updated unit + e2e + docs coverage to reflect the new access pattern and to assert “single read unless accessed”.
File summaries
| File | Description |
|---|---|
| packages/workflow/src/api.ts | Re-exports HookWithLazyMetadata from the public workflow/api surface. |
| packages/core/src/runtime/resume-hook.ts | Implements lazy metadata accessor, updates hook lookup/resume return types, and reuses metadata-derived keys in resumeWebhook. |
| packages/core/src/runtime/resume-hook.parallel.test.ts | Updates test commentary to match the new inline by-token fetch behavior. |
| packages/core/src/runtime/resume-hook.fast-path.test.ts | Updates test commentary for the new “await metadata” behavior on default webhooks. |
| packages/core/src/runtime/get-hook-by-token.test.ts | Adds focused unit tests covering lazy hydration, memoization, I/O behavior, and non-enumerability. |
| packages/core/src/runtime.ts | Re-exports HookWithLazyMetadata from @workflow/core/runtime. |
| packages/core/src/create-hook.ts | Updates API docs to tell users hook.metadata must be awaited outside the workflow. |
| packages/core/e2e/e2e.test.ts | Updates e2e call sites to await hook.metadata when reading custom metadata. |
| packages/core/e2e/e2e-region.test.ts | Updates multi-region e2e assertions to await hook.metadata. |
| docs/content/docs/v5/api-reference/workflow-api/get-hook-by-token.mdx | Documents the new lazy Promise getter semantics and updates examples/return type. |
| .changeset/lazy-hook-metadata-getter.md | Declares a major breaking change and explains the new hook.metadata semantics. |
Review details
- Files reviewed: 11/11 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
a8c1496 to
08b694a
Compare
🧪 E2E Test Results❌ Some tests failed ❌ Failed E2E Testsvercel-http-transport (3 failed)example (1 failed):
express (1 failed):
hono (1 failed):
|
| Passed | Failed | Skipped | Total | |
|---|---|---|---|---|
| ✅ ▲ Vercel Production | 3662 | 0 | 685 | 4347 |
| ✅ 💻 Local Development | 3922 | 0 | 586 | 4508 |
| ✅ 📦 Local Production | 3922 | 0 | 586 | 4508 |
| ✅ 🐘 Local Postgres | 3922 | 0 | 586 | 4508 |
| ✅ 🪟 Windows | 320 | 0 | 2 | 322 |
| ✅ 🌐 Cross-language Conformance | 68 | 0 | 74 | 142 |
| ❌ vercel-http-transport | 820 | 3 | 143 | 966 |
| ✅ vercel-multi-region | 27 | 0 | 0 | 27 |
| ✅ vercel-ws-transport | 399 | 0 | 84 | 483 |
| Total | 17062 | 3 | 2746 | 19811 |
Details by Category
✅ ▲ Vercel Production
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-node | 133 | 0 | 28 |
| ✅ astro-quickjs | 133 | 0 | 28 |
| ✅ example-node | 133 | 0 | 28 |
| ✅ example-quickjs | 133 | 0 | 28 |
| ✅ express-node | 133 | 0 | 28 |
| ✅ express-quickjs | 133 | 0 | 28 |
| ✅ fastify-node | 133 | 0 | 28 |
| ✅ fastify-quickjs | 133 | 0 | 28 |
| ✅ hono-node | 133 | 0 | 28 |
| ✅ hono-quickjs | 133 | 0 | 28 |
| ✅ nest-node | 133 | 0 | 28 |
| ✅ nest-quickjs | 133 | 0 | 28 |
| ✅ nextjs-turbopack-node | 158 | 0 | 3 |
| ✅ nextjs-turbopack-quickjs | 158 | 0 | 3 |
| ✅ nextjs-webpack-node | 158 | 0 | 3 |
| ✅ nextjs-webpack-quickjs | 158 | 0 | 3 |
| ✅ nitro-node | 133 | 0 | 28 |
| ✅ nitro-quickjs | 133 | 0 | 28 |
| ✅ nuxt-node | 133 | 0 | 28 |
| ✅ nuxt-quickjs | 133 | 0 | 28 |
| ✅ python-node | 66 | 0 | 95 |
| ✅ sveltekit-node | 152 | 0 | 9 |
| ✅ sveltekit-quickjs | 152 | 0 | 9 |
| ✅ tanstack-start-node | 133 | 0 | 28 |
| ✅ tanstack-start-quickjs | 133 | 0 | 28 |
| ✅ vite-node | 133 | 0 | 28 |
| ✅ vite-quickjs | 133 | 0 | 28 |
✅ 💻 Local Development
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 134 | 0 | 27 |
| ✅ astro-stable-quickjs | 134 | 0 | 27 |
| ✅ express-stable-node | 134 | 0 | 27 |
| ✅ express-stable-quickjs | 134 | 0 | 27 |
| ✅ fastify-stable-node | 134 | 0 | 27 |
| ✅ fastify-stable-quickjs | 134 | 0 | 27 |
| ✅ hono-stable-node | 134 | 0 | 27 |
| ✅ hono-stable-quickjs | 134 | 0 | 27 |
| ✅ nest-stable-node | 134 | 0 | 27 |
| ✅ nest-stable-quickjs | 134 | 0 | 27 |
| ✅ nextjs-turbopack-canary-node | 141 | 0 | 20 |
| ✅ nextjs-turbopack-canary-quickjs | 141 | 0 | 20 |
| ✅ nextjs-turbopack-stable-node | 160 | 0 | 1 |
| ✅ nextjs-turbopack-stable-quickjs | 160 | 0 | 1 |
| ✅ nextjs-webpack-canary-node | 141 | 0 | 20 |
| ✅ nextjs-webpack-canary-quickjs | 141 | 0 | 20 |
| ✅ nextjs-webpack-stable-node | 160 | 0 | 1 |
| ✅ nextjs-webpack-stable-quickjs | 160 | 0 | 1 |
| ✅ nitro-stable-node | 134 | 0 | 27 |
| ✅ nitro-stable-quickjs | 134 | 0 | 27 |
| ✅ nuxt-stable-node | 134 | 0 | 27 |
| ✅ nuxt-stable-quickjs | 134 | 0 | 27 |
| ✅ sveltekit-stable-node | 153 | 0 | 8 |
| ✅ sveltekit-stable-quickjs | 153 | 0 | 8 |
| ✅ tanstack-start-node | 134 | 0 | 27 |
| ✅ tanstack-start-quickjs | 134 | 0 | 27 |
| ✅ vite-stable-node | 134 | 0 | 27 |
| ✅ vite-stable-quickjs | 134 | 0 | 27 |
✅ 📦 Local Production
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 134 | 0 | 27 |
| ✅ astro-stable-quickjs | 134 | 0 | 27 |
| ✅ express-stable-node | 134 | 0 | 27 |
| ✅ express-stable-quickjs | 134 | 0 | 27 |
| ✅ fastify-stable-node | 134 | 0 | 27 |
| ✅ fastify-stable-quickjs | 134 | 0 | 27 |
| ✅ hono-stable-node | 134 | 0 | 27 |
| ✅ hono-stable-quickjs | 134 | 0 | 27 |
| ✅ nest-stable-node | 134 | 0 | 27 |
| ✅ nest-stable-quickjs | 134 | 0 | 27 |
| ✅ nextjs-turbopack-canary-node | 141 | 0 | 20 |
| ✅ nextjs-turbopack-canary-quickjs | 141 | 0 | 20 |
| ✅ nextjs-turbopack-stable-node | 160 | 0 | 1 |
| ✅ nextjs-turbopack-stable-quickjs | 160 | 0 | 1 |
| ✅ nextjs-webpack-canary-node | 141 | 0 | 20 |
| ✅ nextjs-webpack-canary-quickjs | 141 | 0 | 20 |
| ✅ nextjs-webpack-stable-node | 160 | 0 | 1 |
| ✅ nextjs-webpack-stable-quickjs | 160 | 0 | 1 |
| ✅ nitro-stable-node | 134 | 0 | 27 |
| ✅ nitro-stable-quickjs | 134 | 0 | 27 |
| ✅ nuxt-stable-node | 134 | 0 | 27 |
| ✅ nuxt-stable-quickjs | 134 | 0 | 27 |
| ✅ sveltekit-stable-node | 153 | 0 | 8 |
| ✅ sveltekit-stable-quickjs | 153 | 0 | 8 |
| ✅ tanstack-start-node | 134 | 0 | 27 |
| ✅ tanstack-start-quickjs | 134 | 0 | 27 |
| ✅ vite-stable-node | 134 | 0 | 27 |
| ✅ vite-stable-quickjs | 134 | 0 | 27 |
✅ 🐘 Local Postgres
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ astro-stable-node | 134 | 0 | 27 |
| ✅ astro-stable-quickjs | 134 | 0 | 27 |
| ✅ express-stable-node | 134 | 0 | 27 |
| ✅ express-stable-quickjs | 134 | 0 | 27 |
| ✅ fastify-stable-node | 134 | 0 | 27 |
| ✅ fastify-stable-quickjs | 134 | 0 | 27 |
| ✅ hono-stable-node | 134 | 0 | 27 |
| ✅ hono-stable-quickjs | 134 | 0 | 27 |
| ✅ nest-stable-node | 134 | 0 | 27 |
| ✅ nest-stable-quickjs | 134 | 0 | 27 |
| ✅ nextjs-turbopack-canary-node | 141 | 0 | 20 |
| ✅ nextjs-turbopack-canary-quickjs | 141 | 0 | 20 |
| ✅ nextjs-turbopack-stable-node | 160 | 0 | 1 |
| ✅ nextjs-turbopack-stable-quickjs | 160 | 0 | 1 |
| ✅ nextjs-webpack-canary-node | 141 | 0 | 20 |
| ✅ nextjs-webpack-canary-quickjs | 141 | 0 | 20 |
| ✅ nextjs-webpack-stable-node | 160 | 0 | 1 |
| ✅ nextjs-webpack-stable-quickjs | 160 | 0 | 1 |
| ✅ nitro-stable-node | 134 | 0 | 27 |
| ✅ nitro-stable-quickjs | 134 | 0 | 27 |
| ✅ nuxt-stable-node | 134 | 0 | 27 |
| ✅ nuxt-stable-quickjs | 134 | 0 | 27 |
| ✅ sveltekit-stable-node | 153 | 0 | 8 |
| ✅ sveltekit-stable-quickjs | 153 | 0 | 8 |
| ✅ tanstack-start-node | 134 | 0 | 27 |
| ✅ tanstack-start-quickjs | 134 | 0 | 27 |
| ✅ vite-stable-node | 134 | 0 | 27 |
| ✅ vite-stable-quickjs | 134 | 0 | 27 |
✅ 🪟 Windows
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack-node | 160 | 0 | 1 |
| ✅ nextjs-turbopack-quickjs | 160 | 0 | 1 |
✅ 🌐 Cross-language Conformance
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ python | 68 | 0 | 74 |
❌ vercel-http-transport
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ❌ example | 132 | 1 | 28 |
| ❌ express | 132 | 1 | 28 |
| ❌ hono | 132 | 1 | 28 |
| ✅ nextjs-turbopack | 158 | 0 | 3 |
| ✅ nitro | 133 | 0 | 28 |
| ✅ vite | 133 | 0 | 28 |
✅ vercel-multi-region
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ nextjs-turbopack | 27 | 0 | 0 |
✅ vercel-ws-transport
| App | Passed | Failed | Skipped |
|---|---|---|---|
| ✅ example | 133 | 0 | 28 |
| ✅ express | 133 | 0 | 28 |
| ✅ vite | 133 | 0 | 28 |
📊 Workflow Benchmarkscommit Backend:
Streams
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 133880ms → this run 140316ms (Δ +6436ms, +5%) 📈 CRTT drill-down vs main (RTT distributions & profiles)RTT over stream progress (avg per tenth of stream, bars scaled min→max): RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max): Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max): 📜 Previous results (2)01f8f8dWed, 09 Sep 2026 17:41:20 GMT · run logs
Streams
b89b982Fri, 04 Sep 2026 21:25:30 GMT · run logs
Streams
ℹ️ Metric definitions & methodologyStreams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach. The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000t 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor. |
Hydrating a hook's metadata is a decrypting READ: it needs the owning run's payload keys, and resolving those costs a run fetch plus a `run-key` API round trip (~350ms). `getHookByToken()` did that work eagerly on every lookup that found a metadata-bearing hook, so callers that only wanted `runId`/`token` — and hook resumption, which never reads metadata at all — paid for it anyway. `metadata` is now a getter returning a memoized Promise, the same shape as `run.returnValue`. The lookup is one read again; hydration and the key resolution behind it happen on first access, or never. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-Authored-By: Pranay Prakash <1797812+pranaygp@users.noreply.github.com>
…ion skill, and the resumeHook reference Adds the breaking-change row to the v5 What's new page and puts that page in the sidebar as the first visible entry (the /v5/docs redirect to getting-started is unchanged). Teaches the migrating-workflow-v4-to-v5 skill the `await hook.metadata` rewrite and bumps its version. Points the resumeHook reference at HookWithLazyMetadata, and notes on the World storage page that world.hooks.getByToken() returns raw serialized metadata. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
b89b982 to
e8d1e4d
Compare
`getHookByToken()` and `resumeHook()` return `Hook`, not a separate `HookWithLazyMetadata`: one public hook type whose `metadata` is a lazy Promise, mirroring `Run` for runs. The World-level record from `@workflow/world` is unchanged and is referenced as `WorldHook` inside the runtime. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
VaguelySerious
left a comment
There was a problem hiding this comment.
Left some inline changes for comments/docs.
Note Nate's question about Object.create, would be curious too
…and docs wording Review feedback: the hook record a World returns is a fresh object per lookup and the eager path mutated it anyway, so define the getter on it directly instead of copying it with Object.create(). The changeset is one sentence, and the docs describe hydration as extra network round trips rather than decryption, since not every World encrypts. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
No backport to This is a breaking API change and performance optimization for the v5 line: To override, re-run the Backport to stable workflow manually via |
Summary & Motivation
Hydrating a hook's
metadatais a decrypting READ: it needs the owning run's payload keys, and resolving those costs a run fetch plus arun-keyAPI round trip (~350ms).getHookByToken()did that work eagerly on every lookup that found a metadata-bearing hook, so callers that only wantedrunId/tokenpaid for it anyway — and so did hook resumption, which never reads metadata at all.hook.metadatais now a getter that returns a memoized Promise, the same shape asrun.returnValue. The lookup is one read again; the hydration and the key resolution behind it happen on first access, or never.Expected perf win
Datadog, production, trailing 24h (2026-09-09):
GET /v1/workflow/run-key/:deploymentId— the round trip a lookup no longer pays unlessmetadatais awaitedGET /v2/hooks/by-token— the one read a lookup still paysSo a
getHookByToken()that finds a metadata-bearing hook and never awaitsmetadatadrops from roughly 190 ms to 19 ms at the median (and from ~785 ms to ~264 ms at p95), plus oneruns.geton pre-resumeContexthooks. Callers that only needrunId/token, and everyresumeHook()on a hook that carries metadata, get the whole saving.resumeWebhook()on a webhook withrespondWithis unchanged: it must read metadata, and it still resolves the key exactly once.Picks up @AndrewBarba's "skip metadata lookups in all the places we can" (vercel/eve#3043) and @pranaygp's "+1 here on make the metadata lookup a lazy promise getter in hook".
What changed
getHookByToken()returns aHook(exported fromworkflow/api, the runtime-side hook type whosemetadatais lazy, mirroringRunfor runs): onehooks.getByToken, nothing else. The getter is defined in place on the record the World returned, the same object the eager path used to mutate.resumeWebhook()is the one resume path that must read metadata (respondWithlives there), so it awaits the getter. Behavior is unchanged: a default webhook that stored no metadata still resolvesundefinedwith no key lookup and still letsresumeHookseal to the run's public key, and a metadata-bearing webhook still resolves the run key exactly once end to end — the key the hydration resolved is reused for the payload write.resumeHook()'s returned hook gets the same accessor. Previously it handed back the raw record, whosemetadatawas still the serialized bytes; it now hydrates lazily, which costs nothing when unread.getHookByTokenWithKeyis gone, replaced bywithLazyMetadata+ a directhooks.getByToken. The durable write-then-wake resume path itself is untouched.Breaking change
hook.metadatamust be awaited. Likerun.returnValue, the accessor is non-enumerable, so it is absent from{ ...hook }andJSON.stringify(hook)— that's deliberate: an incidental spread would otherwise kick off hydration nobody awaits, and an unconsumed rejected Promise takes the process down. Forward the awaited value explicitly if you need to serialize it. The World-levelHooktype in@workflow/worldis untouched;world.hooks.getByTokenstill returns raw serialized metadata.Test Plan
New
packages/core/src/runtime/get-hook-by-token.test.ts(9 tests) covers: a lookup costs exactly one read even with metadata present; first access hydrates and memoizes, deriving read-side keys from the stored resume context with no run read; a metadata-less hook resolvesundefinedwith no I/O; the pre-resumeContextfallback reads the run only on access; a hydration failure surfaces onawait, not on lookup; a failed lookup still propagates; spread/JSON.stringifydon't trigger hydration; and a resumed hook carries the accessor without double-wrapping.The existing
resume-hook*.test.tssuites pin the resume behavior that must not move — in particular that a default webhook pays no key lookup and seals, and that a metadata-bearing webhook resolves the key exactly once. All 64 tests across the four hook suites pass, and the fullpackages/coreunit suite is green (2350 passed, 3 expected fail). E2E hook tests were updated for the new access shape.Docs Preview
getHookByTokenresumeHook(returnsHookfromworkflow/api)world.hooks.getByToken()returns raw metadata)Only the v5 pages are updated: the change ships in the 5.x beta, and v4 documents the shipped stable API.
createHook'smetadataoption gains a line about reading it back, which renders from TSDoc on /v5/docs/api-reference/workflow/create-hook. Themigrating-workflow-v4-to-v5skill gains theawait hook.metadatarewrite rule (version 0.2.10). The/v5/docsredirect to getting-started is unchanged;whats-newis only added to the sidebar, in the same position PR #3714 adds it.Server-side follow-up (not in this PR)
workflow-server'sGET /v2/hooks/by-tokenstill resolves metadata eagerly because the SDK sends noremoteRefBehaviorand the server defaults toresolve: for a metadata-bearing hook that is one extra DynamoDB run read (to enforceexpiredAt) plus the ref resolution (in-memory for inlinedbrfrefs up to 64 KB, an S3 GET above that). PassingresolveData: 'none'from the lookup and decoding lazily on first access would remove that too; tracked separately.Expected win: the by-token route's median is already 19 ms, so this is a tail fix, not a median fix. It removes one DynamoDB point read (single-digit ms) from every metadata-bearing lookup and the S3 GET (tens of ms, and the likely driver of the 264 ms p95 above) for metadata above the 64 KB inline cutoff. Expect the route's p95 to move, the median to barely change, and the SDK-side win above to dominate either way.
🤖 Generated with Claude Code